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Abstract —This letter presents a novel Block Bayesian Hypoth¬ 
esis Testing Algorithm (Block-BHTA) for reconstructing hlock- 
sparse signals with unknown block structures. The Block-BHTA 
comprises the detection and recovery of the supports, and the 
estimation of the amplitudes of the block sparse signal. The 
support detection and recovery is performed using a Bayesian 
hypothesis testing. Then, based on the detected and reconstructed 
supports, the nonzero amplitudes are estimated by linear MMSE. 
The effectiveness of Block-BHTA is demonstrated by numerical 
experiments. 

Index Terms —Block-sparse, Bayesian hypothesis testing, 
Bemoulli-Gaussian hidden Markov model. 

I. Introduction 

C ompressed sensing (CS) and sparse signal recovery aim 
to recover the sparse signal, a signal with only a few 
nonzero elements, from underdetermined systems of linear 
equations. In some applications, the unknown signal to be 
estimated has additional structure. If the structure of the signal 
is exploited, the better recovery performance can be achieved. 
A block-sparse signal, in which the nonzero samples manifest 
themselves as clusters, is an important structured sparsity 
Q-ll. Block-sparsity has a wide range of applications in 
multiband signals Q, audio signals Id), structured compressed 
sensing 0, and the multiple measurement vector (MMV) 
model ||8|. The general mathematical model of the block sparse 
signal is 

y = $w + n, (1) 

where $ G j^nxm ^ known measurement matrix, y G 
is the available measurement vector, and n G is the 
Gaussian corrupting noise. We aim to estimate the original 
unknown signal w G , when N <C M, with the cluster 
structure 

w= [wi,... ,Wd^,... ,Wd^_^+i,... , (2) 

^ -Z ^ ^ ^ 

W^[l] 

where w[z] denotes the ?th block with length di which are not 
necessarily identical. In the block partition (|2]i, only k g 
vectors w[i] have nonzero Euclidean norm. 

Given the a priori knowledge of block partition, a few algo¬ 
rithms such as Block-OMP H], mixed £^/£^norm-minimization 
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121 . group LASSO 13 and model-based CoSaMP lH, work ef¬ 
fectively in the block-sparse signal recovery. These algorithms 
require the knowledge of the block structure (e.g. the location 
and the lengths of the blocks) in Q. However, in many 
applications, such prior knowledge is often unavailable. Hence, 
devising an adaptive method for estimating the block parti¬ 
tion and recovering the clustered-sparse signal simultaneously 
remains a challenge. To recover the structure-agnostic block- 
sparse signal, some algorithms, e.g. CluSS-MCMC ID, BM- 
MAP-OMP ITOl . Block Sparse Bayesian Learning (BSBL) 
ifm . and pattern-coupled SBL (PC-SBL) ifT^ have been 
proposed recently, which require less a priori information. 

In this letter, we propose a novel Block Bayesian Hypothesis 
Testing algorithm (Block-BHTA) which uses a joint detection 
of the supports and estimation of the amplitudes. Block-BHTA 
utilizes a Bayesian hypothesis testing (BHT) for the detection 
and recovery of the supports. BHT was first proposed by 
Zayyani et. al. ifTSll in a Bayesian pursuit algorithm (BPA) for 
sparse representations. Recently, BHT with belief propagation 
has been introduced in noisy sparse recovery m. 

Inspired by BPA lfT3l . we adopt a BHT-based approach and 
extend BPA to the block sparse recovery case (Block-BHTA). 
BPA uses the correlations between measurement vector y 
and the columns of matrix and applies a binary BHT to 
obtain an activity rule in which the correlations are compared 
with a threshold. This activity rule is then used for the 
detection and recovery of the supports. Different to BPA, 
Block-BHTA searches for the start and termination of the 
blocks of the supports in the block-sparse signal w. This 
search, performed by the BHT, leads to two ultimate activity 
rules where the correlations between measurement vector y 
and the columns of matrix $ manifest themselves in these two 
activity rules. Hence, the correlations play an important role 
in both BPA and Block-BHTA. In these two activity rules, the 
correlations are compared with two simple thresholds to detect 
and recover the supports. Given the detected and recovered 
supports, Block-BHTA then uses a linear MMSE to estimate 
the nonzero amplitudes. Block-BHTA also uses Bemoulli- 
Gaussian hidden Markov model (BGHMM) ifTSl for the block- 
sparse signals. Using simple tuning updates, Block-BHTA 
utilizes a maximum a posteriori (MAP) estimation procedure 
to automatically learn all parameters of the statistical signal 
model (e.g. the variance and the elements of state-transition 
matrix of BGHMM). The efficiency of the proposed Block- 
BHTA is verified by numerical experiments. 

The rest of the letter is organized as follows. In Section HU 
we present the signal model. In Section Hill the Block-BHTA 
is proposed. Experimental results are presented in Section |IV] 
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Finally, conclusions are drawn in Section |V] 


II. Signal Model 

Consider the linear model of ([T]i as the measurement process 
of an underlying time- or spatial-series which is non-i.i.d and 
block sparse. The measurement matrix is assumed known 
and its columns are normalized to have unit norms. Further¬ 
more, we model the noise in ([T]) as a stationary, additive white 
Gaussian noise (AWGN) process, with n ~ A/” (O, 

To model the block-sparse sources (w), we introduce two 
hidden random processes, s and 9 ifThl . ifTTll . The binary vector 
s G {0,1}^ describes the support of w, denoted S, while 
the vector 9 G represents the amplitudes of the active 
elements of w. Hence, each element of the source vector w 
can be characterized as 


tUj — Si ■ Oi^ (3) 

where Si = 0 gives Wi = 0 for i ^ S and Si = 1 gives wt = 0i 
for 2 G 5. In vector form, Q can be written as w = S9, where 
S = diag(s) G 

To model the block-sparsity of the source vector w, 
we assume that its supports are correlated such that s 
is a stationary first-order Markov process defined by two 
transition probabilities: pio = Pr{si+i = l|si = 0} and 
Poi — Pr{si+i = 0|si = 1}. Therefore, in the steady state, 
Pr{si = 0} = p = and Pr{si = 1 } = 1 - p = 

—^ 7 ^—, which determine the probabilities of the states in 
relation to the transition probabilities. The two parameters p 
and piQ completely describe the state process of the Markov 
chain. As a result, the remaining transition probability can be 
determined as poi = {i^p) ■ length of the blocks of the 
block-sparse signal is determined by parameter pqi, namely, 
the average number of consecutive samples of ones is specified 
by 1 /poi in the Markov chain. 

We further assume that the amplitude vector 9 has a 
Gaussian distribution with 9 ^ Af (O, (jIIm). Hence, the PDF 
of the WiS is given as 


p{wi) = pS{w^) -f (1 - p)Af {wi\0, ag) , (4) 


where ag is the variance of 9. 

Equation (01 is the well known BGHMM which is a special 
form of Gaussian Mixture Hidden Markov model (GHMM). 
The hidden variables Si with the first-order Markov chain 
model in BGHMM allow implicit expression of the block- 
sparsity of the signal w to be estimated. 


III. The Proposed Algorithm 

The proposed Block-BHTA consists of support detection 
and amplitude estimation. Using BHT, we first detect and 
recover the Block-sparse support s. Then, using a linear 
MMSE estimator, we estimate the non-zero amplitudes of the 
detected supports (i.e., estimating 9). 


A. Support Detection Using Bayesian Hypothesis Testing 

We determine the activity of the jth element of the block- 
sparse signal w by searching the start and termination of active 
blocks in w. Toward that end, we assume that Wi is inactive 
(i.e.. Si = 0) and we intend to determine whether Wi+i is 
active (i.e., Si+i = 1). This case is equivalent to searching the 
start of the active blocks. In the second case, we assume that 
Wi is active (i.e., Si = 1) and we intend to determine whether 
Wi+i is inactive (i.e., = 0). This corresponds to searching 

the end of active blocks. Eull details are given below. 

1) Searching The Start of Active Blocks: In order to detect 
the start of an active block we choose one between the hy¬ 
potheses "Hoi : Si = 0, Si+i = 1 and "Hoo : Si = 0, s^+i = 0, 
given the measurement vector y. The Bayesian hypothesis test 
is 

1 p (HoiJ I y) > p (Hoo.i I y), 

0 Otherwise, 

where y is the measurement vector. The posterior probability 
P (Hoi,j I y) is given as 

P {hLoi,j I y) = P (sis^+i = 01 I y) = p (s^ = 0) 

X p (si+i = 1 I Si = 0) X p (y I SjSi+i = 01) 
= p X pio X p (y I SiSi+i = 01), (6) 

where y = j/i VjtUj + ^ ^nd tpj represents the 

jth column of matrix €>. Similarly, the posterior probability 
P (HooJ- I y) is given by 

P (Hooj I y) = P (sis,+i = 00 I y) = p (s, = 0) 

X p (si+i = 0 I Si = 0) X p (y I SjSi+i = 00) 

= p X Poo X p (y I SiSi+i = 00), (7) 

where poo = p (si+i = 0 | Si = 0) = 1 - pio and y = 

Y^j=i i+i V’jti’j +n. Hence, from (|5]l-(l7l), the activity rule 
for Wi+i is 

Pio X p (y I SiSj+i = 01) > Poo X p (y I SjS*+i = 00). (8) 

Assume that we have all the estimates of Wj except for j f 
2-1-1 and we intend to estimate Wi+i. We have 

exp 

p(y|siSi+i = 00) =- 


When SiSi+i = 01, we have y = E^i,j/*,i+i VjWj -f 
<p*+iWi+i + n = + n', where n' = 

V^i+iWi+i + n. Hence, the likelihood p (y | SiSi+i = 01) is 
a multivariate Gaussian with its mean and covariance given 
respectively by 



M 


7: 

II 


(10) 


1=1 


Sy = Cov (n' 

) = + (tIt’i+iT’J+1- 

(11) 


1 

231 






V ( 2710 - 2 )^ 


(9) 
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Therefore, we can write the likelihood function as 

exp (-Hy - ^^yf ^ “ ^^y)) 


p{y I SiSi+i = 01 ) = 


(27r)^det(Sj 


( 12 ) 

Using the matrix inversion lemma (na, p. 571), we can 
express S 


-1 


as 


s;' = a, 


-2 


In — 


V’^+lV’I+l(^n^ 


-(ft)' 

The determinant of Sy can be calculated as 0 


(13) 


det (Sy) = {aef^ det ( ^1^ + ¥’*+i¥’i+i 


= i^e) 


2N 


1 + 


det 


—o^N 




(14) 


Using (I9ll- (fl4l) . the Bayesian hypothesis test in (l8]l can be 
simplified to give the final activity rule for Wi+i as 

ActivitysTART (^i+i) = > Thi,,+i, (15) 

where Thi i+i is defined as 

t2 ’ 


Thi,,+i A 2a^ 



(16) 


and X = y — 4>w — (fiWi — It is seen that in 

the activity rule Activityg-j-^j^rp (wi+i) in (fTsT i the correlation 
between the columns of matrix $ and measurement vector x 
decides between TLqi and TLoq. 

2) Searching The Termination of Active Blocks: The detec¬ 
tion of the end of an active block is performed by choosing 
one between the hypotheses "Hiq ; Si = l,Si+i = 0 and 
^11 : Si = 1, Si+i = 1, given the measurement vector y. The 
Bayesian hypothesis test is given as 


0 p('Hioy I y) >p('Hiij I y) 

1 Otherwise. 


(17) 


Similar to (|6]l, we have p{'HiQj | y) = (1 — p) x poi x 
P (y I s*Si+i = 10), where y = Like¬ 

wise, piTLiij I y) = (1 - p) X pii X p(y I s^Si+i = 11), 
where pn = p (s^+i = 1 | = 1) = 1 - poi and y = 

E m 

rule for Wi+i 

Pm X p (y I SiSi+i = 10) > pii X p (y I s^Si+i = 11). (18) 

Similar to ®, the likelihood function p (y | SiSi+i = 10) is 
calculated as 


jWj -f n. Therefore, we have the following inactivity 


exp - 


p{y\siSi+i = 10) = 


AT 

y - E,=i 




(2^2)^ 


(19) 


^We have used matrix determinant lemma, i.e. det (A + uv^) = 
(l + v^A~^u) det (A), where A is an invertible square matrix and u, 
V are column vectors. 


Also, given s^s^+i = 11, y = ^> 3 ^^ -f 

^Pi+iWi+i + n = Y.f=ij^i+i <PjWj + n', where n' = 
V^i+iWi+i -f n. Hence, the likelihood p (y | SiSi+i = 11) is 
a multivariate Gaussian with its covariance given by (fTTT i and 
its mean by 

M 

fly = iPjWj. (20) 


Also, the likelihood function p (y | SiS^+i = 11) can be eval¬ 
uated as 


exp (-i (y-M;,)^Sy^ (y-My 

(27r)^det(Sy) 

( 21 ) 

where Sy^ and det (Sy) are given in (fOl) and (fT4l) . respec¬ 
tively. Substituting (fT9l l and (ISTT l in ( fTSl l and using (l20l i, the 
final inactivity rule for Wi+i can be expressed as 

InactivityEND (wi+i) = iF^p^^3^pYl, > Th2,j+i, (22) 


where Th 2 y+i is defined as 




and z = y - $w - 

Also, the estimates of the unknown parameters an, erg, p, 
Pio, and poi in (fTdl) and (|2^ are given by the following simple 
updates mi 




||y-$w||2 ^ _ / NV.{y]) ^ _ ||s||(, 


(24) 


Pio 


i=l gt-H (1 ~ St) 

E Af —1 /-I \ 

i=i (l-sU 


,.P01 = 




E m — 1 
2=1 


Si 


(25) 


where E (•) represents the expectation of a random variable. 


B. Amplitude Estimation Using Linear MMSE 

Given the detection and recovery information of the binary 
support vector s by BHT, we complete the estimation of 
the original unknown signal w by estimating the amplitude 
samples of the 6 vector. 

Based on the detected vector s, denoted by s, we obtain 
the linear MMSE estimate ( IITSl . p. 364) of 6 (denoted by 9) 
which is given as 

9 = a^S^^ {allN + y (26) 

where S = diag(s). 

Algorithm [1] provides a pseudo-code implementation of our 
proposed Block-BHTA that gives all steps in the algorithm 
including BHT support detection and amplitude estimation. 
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Algorithm 1 The overall Block-BHTA estimation. 


Input: y, kmax, and e Initialize: Choose G [0.5,1], <7^^^ 


ATKi/?) 


(7^°^ — /h, set difference ^ 1, fc = 0. 

1: while (difference > e and k < kmax) do 
2: BHT-detection: 

3: for i — 0 , ■ ■ ■ , M — 1 do 

4: if Activitygrp^j^rp (rci+i) > Thi,i+i in 051 then 

5: set Si — 1, 

6: else if Inactivity^ND (^■i4-i) > Th 2 , 24 -i in 1221 then 

7: set Si — 0, 

8: end if 

9: end for 

10: LMMSE estimation: S^'') = diag , 

11: = cr^S#^ (cr^Iw 

12: Parameter Estimation: using HM and <251 

13: wf*”) = sf'') 0 6l(''). 

14: Compute the difference — —n— m — fc t— /c + 1 

Ic II2 

15: end while 
Output: w = 


IV. Simulation Results 

This section presents the experimental results to demon¬ 
strate the performance of the Block-BHTA. Two experimental 
results are presented in this section. First, we compare the 
performance of the proposed Block-BHTA with that of BPA 
HD versus SNR. Second, we evaluate the performance of 
Block-BHTA versus number of nonzero blocks and compare 
the performance with some block-sparse signal reconstruction 
algorithms. 

All the experiments are conducted for 400 independent 
simulation runs. In each simulation run, the elements of the 
matrix # are chosen from a uniform distribution in [-1,1] with 
columns normalized to unit f^-norm. The Block-sparse sources 
Wgen are synthetically generated using BGHMM in (|4|i which 
is based on Markov chain process. Unless otherwise stated, 
in all experiments p = 0.9, poi = 0.09 and ag = 1 which 
are the parameters of BGHMM. The measurement vector y is 
constructed by y = $Wgen + n, where n is zero-mean AWGN 
with a variance tuned to a specified value of SNR which is 
defined as 

SNR(dB) 4 201ogio (ll^Wgenll^/ llnll^) . (27) 

We use the Normalized Mean Square Error (NMSE (dB)) 
as a performance metric, defined by NMSE(dB) = 
101ogjg(||w — Wgen ||2 / llwgenll^). where w is the estimate 
of the true signal Wgen. We compare the Block-BHTA and 
BPA at different noise levels. In this experiment N — 192 
and M = 512. We add the Gaussian white noise so that SNR, 
defined in (|27ll, varies between 10 dB and 30 dB for each 
generated signal. 

Figure [T] shows the NMSE (dB) versus SNR for both Block- 
BHTA and BPA. It is seen that Block-BHTA exhibit significant 
performance gain (almost 5 dB) over BPA. 

In the second experiment, we examine the influence of 
the block size and the number of blocks on the estimation 
performance of the Block-BHTA where the block partition is 
unknown. Towards that end, we set up a simulation to compare 
the Block-BHTA with some recently developed algorithms for 



Fig. 1. NMSE (dB) versus SNR for Block-BHTA and BPA. The results are 
averaged over 400 trials. 


O'' 


-0- Block-BHTA 
-HU- PC-SBL 
-Q- EBSBL-BO 
-G- BSBL-EM 
CluSS-MCMC 






/ / 


0.09 0.18 0.27 0.36 0.45 0.54 0.63 0.72 0.81 0.9 

Pn, 


Fig. 2. NMSE versus pQi for Block-BHTA and other algorithms. The results 
are averaged over 400 trials. 


block sparse signal reconstruction, such as the block sparse 
Bayesian learning algorithm (BSBL) mil, the expanded block 
sparse Bayesian learning algorithm (EBSBL) lITTIl . the cluster- 
structured MCMC algorithm (CluSS-MCMC) |j9], and the 
pattern-coupled sparse Bayesian learning algorithm (PC-SBL) 
mil- The size of matrix # is 256 x 512, SNR = 15dB, and 
ag = 1. Recall from Section |II] that the block size and the 
number of blocks of w are proportional to 1/poi- That is, 
when poi is small w comprises small number of blocks with 
big sizes and vice versa. Hence, we vary the value of poi 
between 0.09 and 0.9 to obtain the NMSE (dB) for various 
algorithms. The results of NMSE (dB) versus poi is shown 
in Fig. m As seen from the figure, for poi > 0.36 the Block- 
BHTA outperforms all other algorithms. 

V. Conclusion 

This letter has presented a novel Block-BHTA to recover the 
block-sparse signals whose structure of block sparsity is com¬ 
pletely unknown. The proposed Block-BHTA uses a Bayesian 
hypothesis testing to detect and recover the support of the 
block sparse signal. For amplitude recovery, Block-BHTA uti¬ 
lizes a linear MMSE to estimate the nonzero amplitudes of the 
detected supports. Simulation results demonstrate that Block- 
BHTA outperforms the BPA by almost 5 dB performance 
gain. The Block-BHTA also outperforms many state-of-the- 
art algorithms when the block-sparse signal comprises a large 
number of blocks with short lengths. 
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